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Abstract: This study investigates the effects of using different types of recitation 
sessions with a large-enrollment section of a college ccdculus course on student 
achievement, success rates, and first-year retention. Over a period of six years, 
three different types of recitation sessions were implemented into the large 
enrollment section of a calculus course. During the fall semesters, the results on 
the departmental final examination, the DFW rates, and the one-year retention 
rates of students as STEM majors were examined by the type of recitation session 
used. The three types of recitation sessions studied were: (1) optional mentoring 
sessions at the Math Assistance Center conducted by undergraduate students 
(peer mentors), (2) required mentoring sessions conducted by graduate students, 
and (3) required VGNA (Verbal, Graphical or Geometric, Numeric, and 
Algebraic) Concept activities, which were also coupled with mentoring sessions 
conducted by graduate students. The success of the students in the large 
enrollment section of the course, which included one of the three different types of 
recitation sessions, was compared to the success of students in the small 
enrollment sections of the course (enrollments less than 50 students). The results 
of this study demonstrate methods of raising student success rates in large 
enrollment (lecture-format) courses. 

Keywords: calculus, active learning, concept understanding, college teaching, 
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Introduction 

In order to retain and graduate more STEM majors on campus, this study investigated the 
effects of using different types of recitation sessions with a large-enrollment lecture-format first 
semester college calculus course on student achievement (as measured by the departmental final 
examination), student success rates in the course (as measured by the DFW rates), and first-year 
retention in a STEM discipline (as measured by the number of students who, after taking the first 
semester calculus course in the fall, were retained in a STEM discipline a year later). 

Freshman-level courses, such as calculus, often act like a filter instead of a pump for 
science, technology, engineering, and mathematics (STEM) majors (Steen, 1988). In the United 
States, more than 40% of students fail their first-year mathematics course (Wieschenberg, 1994). 
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Students participating in exit interviews after dropping out of engineering at Purdue University 
cite difficulty with their first mathematics course as a primary reason for leaving (Budny, 
LeBold, & Bjedov, 1998). At Indiana University Purdue University Indianapolis (IUPUI), this 
effect of calculus being a filter is apparent. In recent years, IUPUI has launched an effort called 
Gateway to Graduation to improve student learning and retention in courses with high first-year 
student enrollment. With campus recognition of retention issues in Gateway courses, the 
investigators of this study obtained NSF funding (#0969500), to adapt and adopt best practices, 
in order to increase student retention in, and persistence toward graduation for, STEM majors. 

IUPUI is an urban research and life sciences university, formed in 1969, that serves a 
large (31,000 students) and diverse student population. Approximately 62% of the entering 
students each year are first generation college students, 16% of the student body belongs to a 
minority group, and well over 90% are commuters. Moreover, 32% of students who begin their 
collegiate experience at IUPUI graduate from the institution within six years, and slightly over 
70% of the freshman class are retained to the sophomore year. Over the last few years, the 
institution has focused on improving graduation and retention rates, giving primary attention to 
freshman retention and success initiatives. These initiatives include increased residential on- 
campus housing, a flag system where faculty can alert advisors during the semester of individual 
students facing difficulties, and discipline-specific peer mentoring centers for Gateway courses 
(freshman courses with high enrollments and high DFW rates). 

To meet national goals of STEM student retention and four-year graduation, it is 
desirable to graduate approximately 25% of all STEM majors each year, controlling for 
enrollment growth and other factors. In 2008, 13% (457) of current IUPUI STEM majors 
received a bachelor degree. This is low when compared to the IUPUI campus average of 16% 
and IU Bloomington’s campus average of 20%. This low percent of the IUPUI STEM student 
body that graduates is a result of a retention and persistence problem that begins with the first- 
year retention rate of 63% for STEM majors, compared to 70% for the campus; where calculus is 
a required course in the first-year STEM curriculum. 

As IUPUI has become a more established institution, student enrollment has increased 
faster than the rate of university funding and departmental growth. These issues have led to 
limited resources such as the number of faculty and classrooms available for any one course. 
This has also resulted in increased section size in entry-level calculus courses over time. 
Specifically, the number of students enrolling in first semester calculus in the fall semester 
steadily increased from 271 to 362 students during this six-year study, while the number of 
faculty and the number of rooms available to teach mathematics on campus has remained fixed 
at 6 each semester. This problem increases when considering the total enrollment in all 
mathematics courses, which has grown from 5,928 in 2005, to 7,643 in 2014 (a 29% increase in 
10 years) - making the classroom and faculty shortage growing problem. Therefore, the 
investigators of this study, with support from the NSF’s STEM Talent Expansion Program 
(STEP), explored pedagogical methods to improve student success in large-enrollment sections 
of calculus to, thereby, increase first-year retention and, in turn, raise graduation rates in STEM 
related departments. Specifically, the primary goal of the STEP project is to increase the number 
of undergraduate STEM degrees awarded at IUPUI by 10% per year for five years (an increase 
of an additional 782 STEM graduates by 2015). Therefore, to meet this goal, the investigators of 
this study looked for ways to increase student success in calculus by examining how students 
learn mathematical concepts. 
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Rationale and Design of the VGNA Concept Activities 
Basis in Learning Strategies 

Calculus concepts are often taught primarily through algebraic representations. The 
typical student learns how to manipulate functions algebraically to solve the vast majority of 
calculus problems that would appear on a test. If a student becomes proficient at algebraic 
manipulation, often referred to as “plug-and-chug,” the student will pass the course and be 
promoted to the next course in the sequence. However, this traditional way of understanding 
calculus via algebraic representation is limiting, often missing the big ideas of calculus and, 
thereby, reducing its problem solving nature to plug-and-chug assignments. 

Likewise, a purely algebraic approach may fail to account for the complexity of the 
cognitive processes by which students learn calculus. A student’s way of thinking is influenced 
by how the student develops an understanding of the content. In particular, effective ways of 
thinking develop through a learning approach undergirded by the principle that students need 
intellectual stimulus, or what Harel (1997) has termed, the Necessity Principle. Once an 
intellectual need relative to a mathematical concept is established, an understanding of that 
concept begins. Purely algebraic conceptual representations may fail to adequately establish this 
“need” in all learners. Moreover, students’ ways of thinking influence how they understand 
concepts. A way of understanding a concept in a mathematics course is part of a student’s 
“concept image.” Concepts images — the intellectual representations of concepts whether 
correct or incorrect — are developed through multiple perspectives, and have profound effects 
on comprehension (Tall & Vinner, 1981; Vinner, 1992; Harel, 1997). To ensure that valid 
calculus concept images are constructed, multiple representational forms should be utilized in 
ways that both correct faulty mental pictures of concepts and construct sound mathematical 
frameworks. 

Therefore, the challenge is to teach calculus concepts via multiple representations, so that 
students learn multiple ways of describing a concept (Douglas, 1986; Gehrke & Pengelley, 1996; 
Goerdt, 2007; Pilgrim, 2010; Ross, 1996; Smith, 1994, 1996; Stewart, 2012; Tucker, 1996). 
Through multiple representations the student develops and inculcates a conceptual 
understanding, which will have a profound effect on comprehension and problem-solving ability 
(Harel, 2004). Not surprisingly, current and preceding Calculus education reforms have 
identified the importance of addressing issues in the learning of Calculus from multiple 
representations. For this reason, the VGNA Concept activities were implemented into the large 
lecture section’s recitation periods during Phase 3. Since the 1989 Tulane Conference, Calculus 
reform has been concerned with, amongst other things, “numerical, graphical, and modeling 
problems through the use of computers, open-ended projects, writing, applications and 
cooperative learning” (Ganter & Jiroutek, 2000). The VGNA Concept model seeks to address 
these concerns of the reform movement. 

The individual representations of the VGNA Concept model have been met with some 
success. For example, the discussion (verbal) that occurs in recitation groups regarding calculus 
problems has, in some studies, been shown to improve student achievement (Norwood, 1995; 
Bonsague, 1994; Treisman, 1985). Additionally, the use of writing (placing your verbal 
understandings in writing) in assignments throughout the curriculum has been growing based on 
research-based recommendations and assumptions — first published in a large-scale study of 
British schools by Britton et al. (1975) — that writing is important not only for communication 
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but also for the discovery of ideas and a holistic understanding of a subject. Writing in calculus 
courses has also been shown to promote the construction of conceptual understandings and new 
knowledge (Beidleman, Jones, & Wells, 1995; Cooley, 2002). Student learning has also been 
demonstrated through the implementation of graphical or geometric representations and the 
discussion of those forms (Weller et al., 2003; Monk & Nemirovsky, 1994). The use of graphical 
representation is a key organization and learning strategy of many students; however, in calculus 
except to draw a graph of a function or construct rectangles under a curve to approximate area, 
the geometric arguments taught to students are minimal. Additionally, numeric data or 
representation of problems is almost non-existent in the teaching of calculus; yet most real world 
modeling problems involve analyzing data or measurements in a table. Therefore, the teaching of 
calculus is primarily done with one perspective - algebraic understanding, which the authors of 
this study believe misses the big conceptual understandings of Calculus. 

It was observed during the first two years of implementing required recitations as part of 
the large lecture section of the course (Phase 2) that the recitation instructors, all graduate 
students in mathematics, focused almost exclusively on algebraic representations of solving 
problems with little conceptual discussion. Furthermore, many of the students in these recitations 
were observed in a passive learning mode. The inclusion of collaborative activities in recitation 
grows from educational theory that group work necessarily involves the articulation of goals as 
well as ideas, and that these meta-cognitive activities improve learning and retention (Hillocks, 
1986; Bruffee, 1984). Academic environments that are more competitive than collaborative, and 
those that rely on student learning occurring “spontaneously” during lectures, have been 
associated with “decreased interest or disaffection resulting in little or even regressive change” 
(Bonsangue, 1994, 121). Significant studies have demonstrated that mathematics students placed 
into recitation-type collaborative groups experience greater academic success (Bonsangue, 1994; 
Springer et al., 1999; Herzig & Kung, 2003). Similar successes have also been documented in 
other STEM fields. Collaborative work on physics “tutorials” has been well documented by the 
University of Washington group (McDermott, 1994). In chemistry, Peer-led Team Learning is a 
well-established methodology based on small-group activities performed in recitation sections 
facilitated by perr mentors (Gosser, 1998). In an attempt to redesign the recitations into an active 
learning modality focused on forming conceptual understandings via multiple representations, a 
set of weekly VGNA Concept activities were developed for and implemented in Phase 3. 

The VGNA concept activities were designed to 1) place the student in an active learning 
mode; 2) be done in groups of three or four students; 3) focus on conceptual understandings via 
multiple representations; and 4) take about 40 minutes to complete (half of the allotted time for 
each recitation session). The concept for each week’s VGNA Concept activity was selected from 
the lectures corresponding to the same week. The activities were adopted and adapted from the 
course textbook’s resource manual (Shaw, 2012). An example of one such activity follows. 

Example: The Chain Rule (Chapter 2.5) 

The VGNA Concept activity on the Chain Rule was designed to provide a justification of 
the Chain Rule — by interpreting derivatives as rates of change — and opportunities to use it in 
computing derivatives. During lecture, the students were presented two forms of the Chain Rule: 

(f(g(x)j) ’ = f’(g(x))g’(x) and (dy/dx) = (dy/du)(du/dx). 
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The activity begins with the students in each group discussing whether these two 
equations say the same thing. The students are directed to describe the equations in their own 
words and create a verbal representation (explanation) for their comparison. Each groups’ verbal 
representations will eventually be shared with the entire class. Next the students are given a 
graph of two functions, / and g, and asked to graph the composite function, h(x) = g(f(x)), and 
find its derivative at several points based on the graphical estimation of the slope of the tangent 
line (the algebraic formulas are not given). The students are then asked to justify the Chain Rule 
with a geometric argument. The students are then given a table of numeric values (see Table 1) 
to determine the derivative of various composite functions using/and g. 

Table 1 

Numeric Values of Functions used in VGNA Concept Activity on the Chain Rule 


X 

f(x) 

g(x) 

f’(x) 

g’(x) 

1 

3 

2 

4 

6 

2 

1 

8 

5 

7 

3 

7 

2 

7 

9 


The activity concludes with the students finding derivatives of functions in algebraic 
representations. On all VGNA Concept activities, the algebraic representation is always done 
last. The activities are collected and graded, one score per group. 

A quiz at the end of the recitation session provides formative evaluation to the recitation 
instructor about the level to which students have inculcated the activity’s conceptual 
understandings. Two questions on the quiz that have proven insightful in assessing conceptual 
and notational understanding is: 

(1) Compute: (d/dx) sin v 2 and (d/dx) sin 2 x. 

(2) Where do you stop when using the chain rule? Explain why this is false: 

(d/dx) sin(,C + 3x 2 ) = [cos(,C + 3x 2 )\(4x 3 + 6x)(l2x 2 + 6)(24x)24. 

This VGNA Concept activity on the Chain Rule is one of the more powerful activities in 
the course. The activity causes a lot of discussion and arguments during recitation because it 
challenges the understandings of almost every student in the class, from their use of basic 
notation (i.e., squaring the sine function or its argument), understanding of composite functions, 
to why the chain rule works. Once students have actively studied the verbal, geometric, and 
analytic conceptual understandings, we find they are much less likely to make the common 
algebraic errors on tests (Watt, 2013), leading to lower DFW rates, and higher one-year retention 
rates, as evidenced by the results of this study. 

Methodology 

The study involved 1,956 calculus students who, over a six-year period (three two-year 
phases), enrolled into either a small section of calculus or the large lecture section (see Table 2). 
The small sections of the calculus course had enrollments below 50 students per section (average 
was 46) with a traditional style of classroom presentation and discussion format. The large 
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lecture section of the course had enrollments averaging 92 students per section, with additional 
recitation sections of 25 students per recitation. 

Table 2 


Breakdown of Students by Phase and Section Size 


Phase 

Small Section 
Control 

Large Section 
Treatment 

I 

450 

130 

II 

463 

213 

III 

490 

210 


The recitation format evolved over time, but can be divided into three distinct phases 
lasting two years each. During Phase I (2007 and 2008), the students in the large lecture section 
(like all students in the course) had optional mentoring sessions at the Math Assistance Center 
conducted by undergraduate students (peer mentors). During Phase II (2009 and 2010), the 
students had required mentoring sessions (recitations) conducted by graduate students. A quiz 
was administered during recitation, and the score became part of the course grade. During Phase 
III (2011 and 2012), newly created recitation activities focused on developing mathematical 
concepts via an integrated Verbal, Geometric, Numeric and Algebraic understandings approach 
(VGNA Concept activities). These VGNA Concept activities were collected, graded, and became 
part of the course grade. 

Participants 

The students who enroll in the calculus course are predominately majoring in 
engineering, physical sciences, or mathematical sciences. They either place directly into the 
course from the Compass Math Placement Test (developed by ACT) or completed the pre¬ 
calculus course with a grade of C or better. In the fall semester, the majority of these students are 
freshman, and they are not likely to be familiar with the instructors or their methods of teaching 
when they register for the course. Therefore, the study only examined the fall semester over six 
years in an attempt to control for student selection patterns when registering and to more easily 
determine first-year retention rates. In addition, the student demographics were similar in all 
sections of the course (see Table 3). 

Table 3 

Student Demographics _ 



Female 

Minority 

Freshman 

Engineering 

Science 

Control 

19% 

15% 

68% 

58% 

30% 

Treatment 

17% 

16% 

72% 

59% 

31% 
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Recitation Designs in the Large-Enrollment Section 

As discussed earlier, IUPUI has experienced an increase in the number of students taking 
first semester calculus during the fall semester — from 220 in 2002, to 362 in 2012 (a 60% 
increase). However, the number of sections of the course remained constant at six because of 
faculty and classroom limitations. The department was able to secure one time slot in the lecture 
hall for one section of the course, and the increased enrollment each year was absorbed by this 
lecture hall section. The same tenured faculty member taught the large-enrollment section every 
fall semester during this study. In an attempt to allow students in the large-enrollment section an 
opportunity to ask questions and receive individual attention, various interventions were 
implemented in three phases (each phase was used for a two-year period of time). The recitations 
were led by graduate students in mathematics (16 total), all of whom took the department’s TA 
Workshop required of all graduate students teaching in the department - no graduate student was 
used for a second year in this study (the author was interested in how the VGNA Concept 
activities would be adoptable by new graduate students every year). 

Control Group 

The control group was the small-enrollment sections of the course in aggregate, 
comprised of five such sections every semester, each with fewer than 50 students. The student 
demographics of the control group are comparable to that of the large-enrollment section, except 
for 17 registered graduate students, of which none were enrolled in the large-enrollment section 
of the course. There were eight different instructors of the small-enrollment sections during the 
study (four were associate professors who taught one section every semester of the study) — all 
but two were tenured faculty (each teaching once during the study). Therefore, the authors 
assumed the majority of the teaching in the control group was performed by experienced 
professors who have taught the course multiple times. 

Table 4 


Type of Faculty Teaching Sections of the Course 


Faculty 

Ph.D. Cand 

Assistant 

Associate 

Full 

Control 

1 

1 

4 

2 

Treatment 

0 

0 

1 

0 


Assessment Instruments 

Three measures were used to assess student success in the calculus course: the 
department final examination scores, the DFW rate, and the one-year retention rate after taking 
the calculus course. The DFW rate for the course, or a section of the course, is determined by 
dividing the number of students receiving a course grade of D, F, or W (withdrew from the 
course) by the number of students enrolled in the course at census. A freshman-level or general 
education course with a DFW rate above 30% is considered an at-risk course by University 
College (the freshman advising unit) for students at IUPUI. The majority of freshman-level 
mathematics courses (13 out of 22 in 2010) were considered at risk for freshman at IUPUI. 
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The departmental final examination is written each year by the coordinator of the course 
(who did not teach the course during the period of this study). The exam is a paper-and-pencil 
open-response instrument, with the same number of items testing the same learning objectives 
each year. The instructors of the course do not see the final examination until the day of the 
exam. Students from all sections of the course take the departmental final examination at the 
same time and place during final exam week. The exams are then commonly graded (each 
instructor of the course grades one page of the exam, for all students in the course). 

Results 

For this analysis, an ANOVA test was run, using SPSS (Version 21.0), linear and logistic 
regressions were constructed to compare the control and treatment groups during the three 
phases of the research, no other explanatory variables were considered. The regression models 
looked at each phase of the research to determine differences in final exam scores, DFW rates, 
Calculus II pass rates, and student retention in STEM majors based upon the Calculus I 
enrollment section type. Figure 1 shows the mean final exam score by phase and section type. 
During Phase I, there was no statistical difference between the two sections types (p = .632). In 
Phase II (pc.OOl, t=3.546) and Phase III (pc.001, t=8.042), the treatment group performed 
significantly better than the control group on the course final exam. Additionally, the change in 
final exam scores from Phase I to Phase II (p < .01, F=8.503), and Phase II to Phase III (p = .018, 
F=5.620), demonstrated significant growth within the large enrollment section. 


90 

c 



Phase I Phase II Phase III 
(stdev (stdev (stdev 

24.9/26.1) 21.0/21.2) 21.7/19.4) 


n Small Sections of Course 
■ Large Sections of Course 


Recitation Implementation Phase - Fall Semesters 


Figure 1. Results from departmental final examination, mean (stdev) by type of section. 

When examining the DFW rates, the treatment group has always had a smaller DFW rate 
than that of the control group (see Figure 2). However, this difference was not statistically 
significant (pc.05) until Phase III [pc.01, Exp(B)=1.668]. This finding is also consistent with the 
STEM majors in the course. As with the entire course population, the DFW rates for STEM 
majors in the treatment group were higher throughout each phase (see Figure 3), yet the 
difference was only statistically significant in Phase III [p=.01, Exp(B)=1.641]. 
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■ DFW Rate Small Sections 
DFW Rate Large Section 


Phase 1 Phase II Phase III 

Recitation Implementation Phase - Fall Semesters 


Figure 2. Results from DFW rates by type of section for all majors. 


45 



Phase I Phase II Phase III 


■ DFW Rate STEM Majors 
in Small Sections 

■ DFW Rate STEM Majors 
in Large Section 


Recitation Implementation Phase - Fall Semesters 


Figure 3. Results from DFW rates by type of section for STEM majors only. 

Student success in the follow-up course, Calculus II, was analyzed to see if there was a 
difference between the control and treatment groups during the three phases of implementation. 
Figure 4 shows the Calculus II passing rates of students, from each section type, who passed 
Calculus I. Students from Phase I, who were in the control group had a higher passing rate in 
Calculus II than the treatment group, but the difference was not statistically significant (p=.584). 
For Phase II, students involved in the treatment group had a higher Calculus II pass rate than the 
control group, but as in Phase I the difference was not significant (p=.227). However, in Phase 
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Ill, the difference between the control and treatment group increased creating a statistically 
significant difference [pc.001, Exp(B)=2.215]. 



Figure 4. Pass rates (C or better) in Calculus II based upon Calculus I enrollment size. 

Figure 5 shows the retention rates for direct admit STEM majors from first to second year 
by Phase and section type. While the treatment group maintained a higher retention rate 
throughout all three Phases, the difference was not statistically significant throughout Phases I 
(p=.471) and II (p=.089). However, with the implementation of the VGNA model in Phase III, 
there was a statistically significant difference between the enrollment sections [pc.001, 
Exp(B)=3.87], 
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Discussion 

The purpose of this study was to evaluate the effects of implementing recitations in a 
large-enrollment section of calculus on departmental final exam scores, DFW rates, and one-year 
retention rates of students in STEM majors. Over a six-year period, three phases of different 
recitation formats were studied and the results were compared to the small-enrollment sections of 
the course. 

Department Final Examination Scores 

The results suggest that students performed better on the final examination when 
recitations were required. This is attributed to the fact that student-learning outcomes increase 
with more active learning, which is more likely to occur in recitations than small or large 
enrollment classes. Hence, students required to attend recitations outperformed the control 
groups in Phases II and III and the students in the large enrollment class during Phase I, where 
recitations were optional. 

There was a significant increase in student performance on the departmental final 
examination when VGNA Concept activities were added to the required recitations. 
Pedagogically, this was not surprising. When students are placed in an active role in developing 
their conceptual understandings via multiple perspectives (verbal, geometric, numeric, as well as 
algebraic), they are more likely to inculcate the concept at higher levels and, therefore, more 
likely to solve routine problems with higher proficiency and to apply the concept to new 
situations (increased problem solving). 

In order to determine if the increase in final exam scores during Phase III might be a 
result of the final exam questions evolving to favor the treatment group over time (assessing 
more conceptual understanding of the content), a review of the final exam questions was 
performed to determine if the number, type, or difficulty level of the questions had changed over 
the three phases of this study. A faculty member who teaches calculus at another institution 
performed this review. It was determined that the number of questions did vary each year, but the 
concepts being tested and the method of assessing these concepts was identical every year; in 
addition, the difficulty level of those questions was consistent. 

DFW Rates 

When using DFW rates to measure student success, it is important to note that this rate is 
partly instructor specific; however, the DFW rate of any one instructor of the course did not vary 
more than 6 percentage points. An instructor’s rate tended to be higher when given more than 40 
students in a section, but the correlation of DFW rate to class size was not statistically 
significant. Between the two factors, the DFW rate is more instructor specific and less dependent 
on class size. This probably accounts for why department chairs often assign large-enrollment 
sections to instructors with overall low DFW rates. 

Student success increased as students were placed into more structured and active 
learning modes. Using DFW rates as a measure of student success, the small-enrollment sections 
of the course fluctuated between 35% to 44% over the six-years, averaging 39%. In the large- 
enrollment sections, the DFW rates were the highest (35%) in Phase I when students were 
passive learners in watching the lecture and not required to attend recitations. In Phase II, the 
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DFW rate dropped to 29%, when students were required to attend a recitation focused on placing 
them in a more active learning mode. In Phase III, the DFW rate dropped to 26%, when the 
VGNA Concept activities were implemented. 

An important observation of the study is the comparison of the DFW rates between 
STEM majors and the entire population with respect to the recitation type. For the control group 
(small enrollment sections), the difference in the DFW rate between the STEM majors and entire 
population was never greater than 4 percentage points for all three phases of the study (not 
statistically significant). However, for the large enrollment sections during Phase III (both years), 
the DFW rates were 7 percentage points lower for the STEM majors when compared to the entire 
population in the same section. This finding that STEM majors (more so than other types of 
majors) are more likely to pass the course when the VGNA Concept activities were implemented 
has serious teaching and learning implications for retaining STEM majors. 

Pass Rates in the Next Course 

The effect of the VGNA Concept activities on student success goes beyond the first 
course in the Calculus sequence. All of the students in this study were tracked into the second 
course, where the grades from the two courses were compared for each student. The second 
semester calculus course does have small and large enrollment sections, and the large enrollment 
sections of the second course do have required recitations conducted by graduate students; 
however, the VGNA Concept activities were not integrated into the second course during the 
period of this study. Like most institutions, students are able to register for any section of the 
course - about 62% of the students in the large lecture section in the fall selected the large 
enrollment section for the second course. 

In Phase I and II, there was no significant difference in the pass rate (grades of A, B, or 
C) in the second course of those students who passed the first course, between control and 
treatment groups. However, in Phase III, the difference became significant, with those students 
in the treatment group (large enrollment sections using the VGNA Concept activities) passing at 
a rate 15-percentage points higher than the control group. In addition, the control group 
experienced a 6-percentage point drop in their success rate in the second course from Phase II to 
Phase III (there was no change from Phase I to Phase II). There may be two related dynamics 
that account for these observations in success rates in the second course. First, the Phase III 
students that participated in the VGNA Concept activities developed strong conceptual 
understandings, which have prepared them for future success in the second course. Secondly, the 
students using the VGNA Concept activities inculcated more than a strong conceptual 
understanding of the topics being taught; they also learned new ways of knowing or learning 
knowledge and then used these new ways of learning in the second course. Both of these 
dynamics may have placed the control students at a disadvantage in the second course, where the 
treatment students performed at higher cognitive levels, and the control students found it difficult 
to keep pace with the other students in the class. 

One-Year STEM Major Retention 

One of the important indicators of the number of future STEM graduates is the first-year 
retention rate, which sets the trajectory to graduation. IUPUI, like many institutions nationally, 
has more than half of its first-year STEM majors drop the program by the end of their freshman 
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year. Often Calculus is blamed for this problem by being a filter rather than a pump. Therefore, 
retaining students as STEM majors one-year after the Calculus course was of interest in this 
study. All of the students identified as STEM majors during the first course in Calculus were 
tracked one-year after the course to determine whether they remained a STEM major. During 
Phase I and Phase II, the difference in one-year retention rates was not significant. However, 
during Phase III, the treatment group was 20% more likely (88% versus 68%) to remain a STEM 
major. This is a significant and meaningful result. The Calculus course should adopt more active 
learning strategies with multiple concept representations, not only to lower DFW rates and 
increase future success in the Calculus sequence, but to also retain more STEM majors in the 
first year. 

Additional Notes 

It should be noted that the 2011 control cohort had one section taught by a graduate 
student. This cohort was considered an outlier, affecting two of the control group’s mean scores 
for that year. The mean scores affected were: (1) a lowering of the pass rate in the second 
calculus course, and (2) a lowering of the one-year retention rate. The authors concluded this 
was the result of an inexperienced teacher (graduate student), who either did not focus enough 
attention on the key concepts, did not set expectations high enough, or did not pace the course 
correctly (all common issues for inexperienced teachers). This resulted in too many students not 
being prepared for the next course. This is noted for the possible implications of assigning 
inexperienced teachers to the first course in Calculus. When the data from the 2011 control 
cohort taught by a graduate student was excluded from the analysis, the statistics did not 
significantly change the results or conclusions from Phase III, which combined all the control 
sections over a two year period). 

Implications for Future Study 

From a broader perspective, the primary goal of the STEP project is to increase the 
number of STEM graduates. However, the participants in the first phase of the study are just now 
reaching their six-year graduation date, so it will be several more years before this goal can be 
measured on all students participating in the three phases. In the meantime, students are being 
tracked into second and third year courses where there is an attempt to measure the effectiveness 
of the VGNA Concept activities on student success and retention. Specifically, it must be 
determined if students are transferring these conceptual learning strategies into other math and 
science courses, and if so, whether these strategies are influencing achievement levels. In 
addition, this study did not investigate the possible effects of VGNA Concept activities on 
student success and retention of students in small-enrollment calculus sections. Nor did the study 
take into account the effect of VGNA Concept activities on demographic characteristics of the 
students that participated. Further study on the impact of VGNA Concept activities on gender, 
students of color, stratified age groups, and STEM major would make for interesting research. 

Conclusions 

Over a period of six years, three different types of recitation sessions were implemented 
into the large enrollment section of the calculus course. Although each type of recitation session 
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had different pedagogical models, the time on task (classroom seat time) was the same for all 
phases of this study. During the fall semesters, the results on the departmental final examination, 
the DFW rates, and the one-year retention rates of STEM students were examined by the type of 
recitation session used with the large enrollment section. The three types of recitation sessions 
studied were: (1) optional mentoring at the Math Assistance Center conducted by undergraduate 
peer mentors, (2) required mentoring conducted by graduate students, and (3) required VGNA 
(Verbal, Graphical or Geometric, Numeric, and Algebraic) Concept activities, which were also 
coupled with mentoring conducted by graduate students. The success of the students in the large 
enrollment section of the course, which included one of the three different types of recitation 
sessions, was compared to the success of students in the small enrollment sections of the course. 
The effects of using each type of recitation session on raising departmental final examination 
scores, lowering DFW rates, and raising one-year retention rates were examined. This study 
found that the most significant increases in student learning outcomes and one-year retention 
rates clearly occurred in the third type of recitation, in which students were taken out of their 
passive learning environments and integrated into environments of active learning (e.g. group 
work and collaborative learning) where, through the use of the VGNA Concept activities, 
knowledge construction occurred. 

The increase in retention, persistence, and student graduation in STEM-related majors is 
critical to the nation’s economic well-being. It is evident to educators, employers, and politicians 
that more students need to become successful in flowing through the STEM pipeline. As part of 
this effort, the mathematics education community has been working on making calculus a pump 
and not a filter in the STEM pipeline for more than 25 years; however, too many students today 
are still not being retained in STEM disciplines because of their initial calculus course. This 
study has demonstrated that, even in large-enrollment sections of calculus, the implementation of 
highly structured recitation activities that focus on placing the student in an active role of 
developing their conceptual understandings of mathematics via verbal, geometric, numeric and 
algebraic representations can increase the student success rate in calculus and increase the first- 
year retention rate for STEM students. In time, it is expected that these increases will have an 
effect on the number of students completing a STEM degree — flowing from the pipeline and 
into the economy. 
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